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standards . An evaluation was conducted to determine how the indicators and 
the developed measures and standards are used for program improvement, 
curriculum and instruction, staff development, enhancement of student 
performance, and workforce preparation. Staff from each of the state's 225 
programs were invited to participate. A self-study guide was developed so 
that adult education directors could evaluate their programs in the context 
of the indicators and standards, and this guide became the basis of the 
interview protocol used in interviews at 87 program sites. Findings are 
grouped with each of the eight indicators. The field data strongly suggest 
that the eight indicators of program quality and the ancillary measures and 
performance standards form the framework of most, if not all, adult education 
programs in the state. Program directors have been striving to make their 
programs more responsive to student needs and to adhere to sound management 
practices. Findings also highlight the efficiency and effectiveness of 
programs that are part of a coordinated service delivery network and those 
that offer a continuum of comprehensive services. Appendixes list the program 
sites, provide interview results, and present a discussion of performance 
measurement, reporting, and improvement systems. (Contains 20 references.) 
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"By the year 2000, every adult American will be literate and possess the 
knowledge and skills necessary to compete in a global economy and 
exercise the rights and responsibilities of citizenship. " 



Goal Six, National Education Goals 



Background: 

The New Jersey State Department of Education is providing leadership, technical 
assistance, pohcy and funding support, and accountability to adult education and hteracy 
program providers in the state in order to support the federal efifoit to attain the sixth 
National Education Goal that all Americans will be hterate by the year 2000. Despite 
^America's preeminence as a world power and economic leader, the National Institute for 
Literacy points out that twenty million adult Americans lack the basic literacy skills which 
are necessary to be producthe workers, full partners in their children''s education, and 
citizens involved in building strong communities. 

Trends from New Jersey parallel those witnessed at the national level. 

■ More than 1.5 million adults in New Jersey do not have high school 
diplomas. 

■ New Jersey embraces nearly 30,000 immigrants yearly, most of whom do 
not speak English and some who are not hterate in their native language. 

■ Over 15,000 youth drop out of New Jersey’s high schools each year. 

■ Approximately 60 percent of individuals at the lowest levels of hteracy lack 
full-time employment as defined in the National Adult Literacy Survey. 

The New Jersey State Department of Education in its leadership and capacity-building role 
has embraced the development of a state accountabihty, evaluation, and improvement 
system that is premised on indicators of program quality, measures of performance and 
staridards. This flexible framework for systemic reform provides a mechanism for 
focusing on the results of investments in hteracy and on improving the return on those 
investments. 
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This statewide system — accountability, evaluation and improvement — will allow New 
Jersey to build capacity for its adult education and literacy programs across the State by: 

O focusing on efforts to achieve not only local goals but state and national 
goals as well; 

© measuring progress by reviewing results and impact not process; 

© linking literacy to broader state goals that empower other state agencies to 
maximize resources, share information and streamline reporting in an effort 
to serve the client; and, 

O ensuring continuous improvement of programs that are effective and 
achieve the desired results, discontinuing those which don’t work and 
modifying those which require alternative strategies for success. 



Federal Role in Promoting 



.Accountability; 



The Adult Education Act (P.L. 100-297) established the federal role in supporting basic 
skills instruction to educationally disadvantaged adults through the funding of local 
instructional programs for adults in an array of areas including adult basic education 
(ABE), adult secondary education (ASE) and English as a Second Language (ESL). 

The Act also promotes quality in state and local programs through requirements for 
program evaluation and accountability. Amendments to the Act, embodied in the National 
Literacy Act of 1991, further required that program indicators be established by which 
state and local programs receiving federal funds could evaluate the effectiveness of their 
efforts. 



The indicators of program quality can be used for: 

Program Improvement: To promote the development of quality adult 
education programs. 

Program Accountability: To help ensure local programs provide a 
consistent level and quality of instructional services. 



Program Impact: To identify program and staff characteristics that 
correlate with quality of instructional services. 
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Public Policy: To demonstrate to other progr ams , legislators, policy 
makers and the general public the role and effectiveness of adult 
education programs. 



New Jersey's Implementation of Program Quality Indicators. Measures 
and Standards: 

In the winter of 1992, adult education program directors and staff, as well as adult literacy 
volunteer organization coordinators and literacy volunteers, in conjunction with personnel 
from the New Jersey State Department of Education, initiated the development and 
adoption phase of New Jersey's indicators of program quahty. They also identified 
measures which would be used to determine the quantitative level of program 
performance. 

The eight mdicators of program quahty which were vahdated and adopted are hsted 
below: 



Educational Gains 



Indicator 1: Learners demonstrate progress toward attainment of basic 
skills competencies that support their educational needs and goals. 

Indicator 2: Learners progress according to their abihties m the 
instructional program or complete educational program 
requirements that allow them to continue their education or training 
or achieve their personal educational goals. 

Proeram Planning 

Indicator 3: Based on a participatory planning process, the program 
develops, ircplements, evaluates and revises, as needed, written 
program goals. 

Curriculum and Instruction 



Indicator 4: The program has curriculum and instruction sensitK e to 

mdividual student and group learning styles and the varying levels 
of student need. 
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Staff Development 

Indicator 5: The program has a staff development process that considers 
the specific needs of its stafl; offers opportunities for training in the 
skills necessary to provide quality instruction and for the utilization 
of skills learned. 



Support Services 

Indicator 6: The program identifies support service needs and the 
agencies which address these needs, provides infonnation to 
students as needed, and promotes student access to these agencies 
through informal linkages with service providers as a means of 
enhancing student participation in the program. 

Recruitment 



Indicator 7: The program recruits the population in their service deli\ ery 
area as identified in the Adult Education Act as needing literacy 
services. 

Retention 



Indicator 8: Students are encouraged to remain in the program long 

enough to meet their educational goal(s). The program has a plan 
which addresses retention and implements the plan as needed. 

The next phase of the process involved establishing performance standards defining levels 
of acceptable attainment for each of the eight indicators. During the winter of 1993-94, 
representatives of the adult education community again came together in a developmental 
cycle to begin the process of establishing performance standards. A steering committee 
initiated the first drafts of the document, which detailed standards for each of the 
indicators. 



Definitions: 



The following set of definitions was used as the organizing rubric during the standards 
development phase: 
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Indicator: A quality indicator is a variable that reflects effective and 
efficient program perfor man ce 

Measure: A measure is defined as the data used to determine the 
quantitative level of performance. 

Performance Standard: A performance standard defines a level of 
acceptable performance in terms of specific numeric criterion. 

Evaluation: The process of gathering data for decision-making. 

Process Evaluation (Formative Evaluation): Serves the day-to-day 

decision-making needs required to carry out a program. It provides 
feedback to the producers and managers of a program, so that they 
can monitor the operations and detect and predict potential 
problems in design or implementation. 

Product Evaluation (Summative Evaluation): Serves to measure and 
interpret program attainments; it focuses on the extent to wliicli 
goals had been achieved. Product evaluation may also include 
assessments during the course of the program. 

Roundtables were hosted in 1993 to seek the input of the adult education communitv in 
New Jersey. Staff from each of the 225 programs funded through the New Jersey State 
Department of Education were invited to participate in discussions regarding the standards 
drafted by the steering committee of practitioners. Preliminary drafts were revised a 
number of times based on the suggestions received. 

In the early spring of 1994, the performance standards document was put in a final draft 
form and mailed to all adult education directors and stafl^ as well as to adult literacy 
volxmteer organization coordinators and literacy volunteers for their final review and 
critique. 

Public hearings were held on the performance standards so that all interested educators 
had an opportunity to comment. After the public hearings were conq)Ieted, the 
performance standards were shared with the New Jersey State Board of Education and 
with the State Employment and Training Commission (SETC) prior to their formal 
submission to the United States Department of Education's Office of Adult and Vocational 
Education. 



In summary, the development of indicators of program quality, measures of performance 
and standards for adult education programs mirrors the development of core curriculum 
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content standards which the State Board of Education has endorsed and which have 
become part of the new chapter of State A dminis trative Code entitled. Standards for 
Student Achievement (N.J.A.C. 6:30-2.6). 



Rationale for the Study: 



Adult Education has faced increasing demands to demonstrate effectiveness and value of 
the services and instruction offered. The effort at the national level to strengthen 
accountability requirements led to the development of program quahty indictors to be used 
for program evaluation, accountability and improvement purposes. New Jersey mirrors 
the national trend and is faced with similar accountabihty demands at the state level. The 
indicators therefore ser\e the state quite naturally as an assessment tool to evaluate 
program effectiveness, to improve overall program quahty, to make program funding 
decisions, and to identify' technical assistance needs. 



Tlie New Jersey State Department of Education tlnough the implementation of the 
Indicators of Program Quality: Measures of Performance <&: Standards, is now able to 
assess areas of program strength and weakness at the state level and can respond by 
adjusting its delivery system and technical assistance support to areas requiring 
improvement. These indicators represent a critical step in efforts to promote quahty in 
programs that serve as the foundation for ensuring that every New Jersey citizen wih have 
the necessary skihs and competencies to become a contributing member in New Jersey's 
economy and a valued contributor to the nation's economic growth and development. 



Recognizing the need to determine the degree to which New Jersey Adult Education 
Programs were moving forward in their implementation of the indicators, the State 
Director of Adult Education requested that a field-based study be conducted to determine 
how these indicators, measures and standards were utilized for the purposes of program 
improvement, curriculum and instruction, staff development, enhancement of student 
performance and workforce preparation. 



Methodology: 



An evaluation study was undertaken through the Office of Specialized Populations in 
cooperation with the Adult Literacy Enhancement Center (ALEC) at the Academy for 
Professional Development-Central in the late fall of 1996. It was determined that the 
focus of the field-based smdy would be Adult Basic Education (ABE) Programs and 
English as a Second Language (ESL) Programs which had been funded during the 1996- 
97 program year. Tlie express purpose of the study was to assess individual program 
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implemeatatioa of the Indicators of Program Quality, Measures of Performance & 
Standards for program improvement purposes at the district and state level. The study 
itself was not part of the State’s monitoring process and no fiscal considerations or 
decisions were attached to the study. A total of 87 programs were visited. 

Prior to the implementation of field visits, a self-study guide framed around the eight 
quality indicators was developed, reviewed and refined. Attention was focused on the 
guide’s content vahdity. The self-study guide was prepared so that adult education 
directors would have a tool to use to review their local adult education programs in the 
context of the indicators. The self-study guide later served as the basis for the interview 
protocol which was used during on-site visits conducted by staff from ALEC. The self- 
study guide and interview protocol are provided in the Appendix to this report. The on- 
site visits consisted of interviews wnth the adult education director and appropriate 
teaching staff; review of student portfohos and records; as well as obsen ation of on-going 
classes, if feasible. 

In advance of the field visits, the State Director of Adult Education hosted three regional 
briefing sessions in October of 1996 to apprise the adult education communiry' of the field- 
based evaluation study which was about to be initiated. The Director reinforced the 
rationale for the study and re-emphasized the fact that the findings of the study were not in 
any way connected to program monitoring or fimding. These briefings allow ed for a forum 
at which directors could discuss any concerns or issues which they wished to raise 
regarding the projected study or any other adult education related issues. 

After the briefing sessions had been conducted, site-based field visits were scheduled. 
Over the course of the study, site visits during hohday periods, school vacations, and the 
weeks immediately preceding the due date for the 1997-98 Request for Proposal: Adult 
Basic Skills Grant Program (ABE) ayid Adult Secondary Completion (GEDj could not be 
arranged. Tw o staff from ALEC conducted the interviews and gathered the programmatic 
data. Data collection procedures were refined prior to implementation. An interview 
protocol was conq)Ieted and additional post-interview field notes were prepared. 
Su mmar y data from the interview protocol can be foimd in the Appendix. Ancillary 
programmatic data, recommendations and suggestions were also provided to the field staff 
during the course of the field visits. 

The following sections of the report. Focus on the Program, detail: 

O intention and philosophy of the adult education program given the federal 
mandate through the National Literacy Act of 1991; 

© adults targeted and served through the provision of programs: 
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© adult education progr ams looks like across the State; and 

O staff and the administration of the progr am 



Focus on the Program: What is the intention of the program? 

The National Literacy Act of 1991 (P.L. 102-73) provides support to the States to 
improve educational opportimities for adults who lack the level of literacy skills which are 
requisite to effective citizenship and productive employment. The Act (Sec. 311) also 
assists the states to e.xpand and improve the current system for delivering adult education 
services including services to educationally disadvantaged adults, and to encourage the 
establishment of adult education programs that will— 

O enable these adults to acquire the basic educational skills necessary for 
literate functioning; 

© provide these adults with sufficient basic education to enable them to 
benefit from job training and retraining programs and obtain and retain 
productive employment so that they might more fully enjoy the benefits and 
responsibilities of citizenship; and 

© enable adults to continue their education to at least the level of completion 
of secondary school. 

Given the state’s demographic profile, immigration both legal and illegal and in-migration 
data for New Jersey, it is clear that New Jersey has significant numbers of adults requiring 
adult education programs and services. These programs focus on: basic literacy services 
for adults whose skills are below the eighth grade level; services designed to prepare 
students to obtain a high school equivalency diploma and services for adults with limited 
English proficiency. Through the provision of these types of programs, adults can become 
full participants in the economic fabric of their communities and the State. 

It is clear that all citizens must become viable and productive contributors to the economic 
well-being of the State but also they must be empowered to provide for the educational, 
emotional and physical needs of themselves and their families. Breaking the cycle of 
marginal functional literacy and ilhteracy for these adults will assist in shattering the ties of 
intergenerational ilhteracy and poverty. Adult programs coupled with transitional 
programs leading to gainful employment are the threshold to economic self-sufficiency and 
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to the decline in the need for public assistance and dependency for those adults who are 
not presently members of the workforce. 





Focus on the Program; Whom does the program target and serve? 

The purpose of the Adult Education Act is to improve educational opportunities for 
“adults who lack the level of literacy skills requisite to ejBfective citizenship and productive 
employment.” Adult education programs whether in basic skills, English as a second 
language, high school completion or citizenship are the stepping-stones to economic 
independence and self-sufiBciency for adults most in need. Adults who qualify for 
participation in adult education programs: 

O are 16 years of age or are beyond the age of compulsory school 
attendance; 

© are not enrolled in secondary school; 

© lack sufiScient mastery of basic educational skills to enable them to function 
effectively in society or who do not have a certificate of graduation from a 
school providing secondary education; 

O are not currently required to be enrolled in school; and 

© lack the mastery of basic skills resulting in an inability to speak, read, or 

wnte the English language which constitutes a substantial impairment of 
their ability to get or retain employment commensurate with their real 
ability, and thus are in need of programs to help eliminate such inability 
and raise the level of education of such individuals with a view to making 
them less likely to become dependent on others. 

P.L 1G2-73, Sec. 312 

Adults eimolled in adult programs face many additional barriers to participation. While 
some may be employed, they in feet might have two marginal jobs and consequently have 
associated problems of finding appropriate child care and adequate personal time for class 
participation. The lack of transportation is another hardship often faced by students. This 
clientele may be additionally burdened by substance, spousal or sexual abuse; family 
problems; social issues; or health concerns. 

The profile of an adult ESL student varies. While some may be well-educated, afiduent 
and self-employed, other program participants are unemployed, confi-onting welfare issues 
and, in some cases, homelessness. In addition, adults may be immi grants or refugees who 
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must deal with the newly enacted immigration reforms and the ensuing concerns brought 
about by these reforms. 

For non-native speakers of English, additional obstacles and barriers are faced by students 
since in some cases these adults are not literate or proficient in their native language. 
Learning English, when one has no functional gra mma r or syntax in one's native language, 
can be a daunting task. The problem is even more complex for native-born Americans 
who speak a language other than English in the home and are functionally illiterate in both 
English and their native language. 

Research indicates that a portion of adults enrolled in pro^ams may also have learning 
disabilities. Nightingale et al. (1991) estimate that between 20 and 30 percent of 
economically disad\ antaged adults--and as many as 50 to 80 percent of ABE participants 
may be learning disabled. The National Adult Literacy Survey (NALS) results show that 
about a quarter of persons in the lowest skill level indicate that they have a disability. 
Although the incidence of learning disabilities appears quite significant fi"om these data, 
experts do not agree on whether it is feasible, necessary or even desirable to distinguish 
between adults with learning disabilities and those adults who are low- literate. There Is no 
consensus on the impact of learning disabilities on achievement in adult education 
programs. 



Focus on the Program: What does the program look like across the 
state? 



Adult education programs are implemented in various venues across the state. In some 
settings, both ABE and ESL programs are provided, depending on client need, and in 
other settings only ABE or ESL is offered. Programs vary from those totally integrated 
into settings such as a community learning center, adult high school postsecondary 
institution, vocational technical school etc., to programs which stand alone, are on the 
margin and generally ancillary to other programs provided by the sponsoring agency. 
Maintenance of the program may reflect the dedication and commitment of an adult 
director and his/her colleagues to the educational needs of the clients served by the 
program. 

Adult education programs are offered through a wide variety of agencies including public 
schools, vocational technical schools, community colleges, educational service 
commissions, co mm u nit y-based organizations, labor unions and special agencies which 
serve clients with disabilities. These agencies for special needs clients typicallv include 
Easter Seals of NJ/Associations for Retarded Citizens (ARCs), rehabilitation centers and 
cerebral palsy centers. Programs may be offered in traditional school or college settings, 
as well as in churches, synagogues, libraries, community agencies and other social service 
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settings where access by the clients in need of the program services is fecilitated by a 
convenient location. Some program delivery sites are well-equipped while others are 
hampered by facilities which are older and less well maintained. In some classes space 
constraints and overcrowding were issues. 

The adult education program overall is a program that is operated on the margin in most 
sites. Adult education may not always be in concert with the mission of the sponsoring 
agency. Space availability during the day may conflict with the operation of other day 
programs. Even though this is the case, there seem to be significant contributions by 
sponsoring agencies. These in-kind contributions include classroom space, supplies, 
secretarial assistance, postage, printing, janitorial support, and salaries for security guards 
in some of the inner-city locations. 

It appears to be more beneficial to an adult education program to be integrated into a 
larger functioning entity such as a co mmuni ty learning center, adult high school, 
postsecondary institution or vocational technical school, etc. Resources can be leveraged 
and there may be more options and opportunities a\ailable for the students who 
participate in the program. Adult education programs must connect to a greater uni\ erse 
of support and job networking in order for these programs to be considered effective. 
Students, who are not presently employed, must be helped to transitional bridge programs 
and training programs which can then lead to jobs and the elimination of the need for 
public assistance. 

In some sites, new computer technology is available for student use; however, this is not 
true in most cases. Space for classes as well as storage for materials and books is also an 
issue for most programs, with related problems of centralized recordkeeping and 
maintenance of files. . 

ABE and ESL services may be offered during the day, and/or afternoon and/or evening 
depending on the size and scope of the program In some cases, Saturday classes and 
summer programs have also been provided at a satellite site. Some programs are full-time 
day programs delivered at learning centers but the greater majority of the programs are 
operated on a part-time basis. While program horns vary across the state, typically, 
programs are offered in the evenings. Most programs offer at least the minimum number 
of hours called for in the grant guidelines. While funding may be one of the issues 
impacting program delivery, other issues such as staffing, sectuity, availability of substimte 
teachers, transportation, child care and the provision of counseling services are also of 
concern. 

It should also be noted that in addition to those grant funded ABE and ESL programs 
wliich are provided across the state, sites might also be offering programs such as Adult 
Secondary Completion/GED, GED Testing Centers, and Evening Schools for Foreign 
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Bora Residents (EFB). These additional programs are either supported through state 
funds or are fee-driven. Sites also provided supplementary ABE and ESL programs 
supported through other funding sources. 



Focus on the Program: Staff and administration of the program 

Directors of local adult programs may have this program responsibility on a full-time basis 
while others manage additional federal and/or state grants, alternative education and 
community education programs. Likewise, staff who teach in the program may have full- 
time jobs during the day if they teach at night or have other part-time supplementaiy' jobs 
to eiiliance their teaching salary. Teachers in the adult program may extend themselves 
be> oiid theh educational roles to function as counselors, mentors and problem-solvers for 
students who are in their classes. It is not unusual to hear of staff who have assisted their 
students in finding jobs, obtaining drivers' licenses, working through immigration issues 
and other problems. In some cases, staff even transport students to and fi-om class so that 
their students can save on transportation costs. 



Staff hold certification in a wide variety of areas ancillary to adult education. Teachers in 
adult education programs range from full-time to part-time with a majority of staff 
employed in a part-time capacity. Due to part-time status, variation also e.xists in the 
benefits received by part-time teaching staff. Personal co mmi tments to primarv jobs. 
tamiUes. continuing education, etc. may influence the ability of part-time staff to 
participate in adult education-related staff development opportunities. The part-time status 
of adult education staff has made professional development among teachers a major 
challenge for the state and for local adult education programs. 



.Adult education programs also rely on volunteers to provide program support. 
Volunteers in the program are used mostly as tutors. While programs may employ full- 
time counselors, the bulk of adult education programs rely upon counseling staff from 
other programs; part-time counseling staff supported through adult education programs 
are available, albeit inconsistently, to program participants. 



Results: 




Educational Gains 

Indicator 1: Learners demonstrate progress toward attainment of basic skills 

competencies that support their educational needs and goals. 
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Findings: 

Data from the study indicate that students in the ABE and ESL programs are routinely 
assessed prior to or upon entry into the program within the first ten hours of enrollment. 
In most cases, students were assessed at the time of their registration for the program in 
order to determine appropriate placement. A number of different tests are used for 
assessment, ranging from a wide variety of informal reading inventories for Level I ABE 
students such as the Slossen, the Wide Range Achievement Test (WRAT) and the Adult 
Basic Learning E.xam (ABLE) to the Tests of Adult Basic Education (TABE) for Level II 
and Le\ el III ABE students. Almost universally, the New York State Placement Test 
(NYS Place) is used for entry level assessment for ESL students although one program 
reported use of the Basic English SldUs Test (BEST) and another indicated their use of the 
Ullman ESL Acliievement Test for Beginning ESL Students. Curricula mav not be 
aligned with assessment results despite the use of appropriate testing and assessment 
practices. 



Directors who utilized computer-based recordkeeping software were more likeK’ to have 
immediate access to student achievement, enrollment and other demomaphic data for 
reponing and recordkeeping purposes. A variety of software is currently used across the 
state ranging from commercially developed products to software designed by local agency 
personnel. 

Appro.ximately 79 percent of the programs are complying with the reassessment 
requirement for their students after 50 hours of class participation. Some directors 
indicated that the concern is not with reassessment after 50 hours but indeed with 
retention of their students to that level of attendance. Program directors indicated that 
they felt that their students were making progress within level after program participation. 
Program directors’ comments were supplemented by discussions with teachers and 
program staff who attested to the fact that they had noted progress, iirqirovement and 
gains in computation, communication and life skill competencies. However, data to 
accurately assess the progress was not available with any degree of consistency. 

Directors and staff indicated that 60 or 75 percent inqirovement was high but attainable if 
they could sustain the participation of their students. Only three program directors, 
actually calculated the percent of growth of their students from the pre-test experience to 
the post-test experience. It was noted that, depending upon level and where a student was 
placed at the begmning of the program, dramatic results might be achieved even though 
this achievement gain was less than 60 percent or 75 percent. For example, if a barely 
fimctionally literate Level I student could exit the class being able to write his or her name. 
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recognize simple words and produce single sentences this might be a noteworthy gain yet 
falling below an anticipated percentage. 

Reviewing student records, it is clear that the individualized educational plans contained in 
ponfolios form the cornerstone for an individuars programmatic instruction. Portfolios 
varied in scope, rigor, quality and utility with the contents being non-uniform from 
program to program. Most contained, however, student biographical information, 
attendance data, standardized test data and other performance measures. When available, 
portfolios were enhanced by individual student needs assessment data, student plans, and 
multi-faceted examples of student work including writing samples and other performance- 
based or authentic assessments, student comments and feedback relatKe to program 
participation. These tyjjes of data were generally the exception rather than the rule. 

Some program directors developed individual skill specific folders for each level of their 
program. For example, one ABE program had three unique color-coded portfolio jackets 
detailing a liierarchy of skills to be mastered. The same was true for the .AJBE program. 
Some portfolio folders also contained information which would later be useful to program 
directors in reporting student outcome data to their local boards or to the New Jersev 
State Department of Education. 

Recomm en dations: 

O Establish and utilize portfolios detailing student biographic 

information, skill needs, test information and examples of student work. 

© Strive toward the implementation of computerized recordkeeping to allow 
for standardized reporting and analysis of administrative data. 

© Improve procedures and documentation of data to measure student 
progress and achievement. 

O Align local program tests to program curriculum. 



Educational Gains 

Indicator 2: Learners progress according to their abilities in the instructional 

program or complete program educational requirements that allow them to 
continue their education or training or achieve their personal educational goals. 
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Findings: 

Results of discussions with program directors and staff and a review of data from student 
portfolios indicated that students are making progress and demonstrating gains as a result 
of program participation. Categorically, directors were emphatic in their assertion that the 
program has benefited their students. Students who had participated in the program for at 
least 50 hours of instruction progressed within level Students also moved from one level 
to the next especially from Level II to Level HI. It should be noted that progress from 
Level I to Level 11 within a program year may be difficult to achieve since a great majoritv 
of the students in this level are at the very lowest level of literacy or academic 
achievement. 

It students participated in the program for at least 100 hours, their gains were more 
noteworthy according to program directors. Mikulecky and Lloyd (1993) also point out 
that 100 hours plus of instruction and program participation are needed for adult learners 
to acliieve what the\- define to be a one-year gain in reading level ability. Others (Venezky 
et al. 1993) challenge this view indicating that it is difficult to document a relationship 
between hours of instruction and gains. Moore and Stavrianos ( 1993) point out that data 
fioin a recent national evaluation of Even Start support the' notion that adults gain in 
literacy as hours of instruction increase. Interestingly, these gains seem to occur at a 
slower pace as the hours of instruction increase. The data from the Even Start study 
support the time o)i task (ihe more you do something the better you will be at it) adase 
and concur with conventional pedagogical wisdom, practice and intuition. 

With the e.xception of ABE programs provided to students who were developmentally 
disabled in sheltered workshops or adults who may have special needs, directors felt that 
their students moved from level to level as outlined in the interview protocol; however, 
only three had culled the data and actually analyzed the percentage of movement. Adults 
who did not show unprovement wdthin Lev'el I after 50 hours tended to reduce the overall 
percentage showing gain. In the case of ESL students, directors indicated reluctance of 
students to move to the next level, even though their test scores and overall achievement 

would have warranted progress, since they had formed social ties with fellow students and 
their families. 

With respect to the program’s ability to meet students’ individual personal educational 
goals, directors and staff were emphatic that students were meeting if not exceeding their 
own individually stated goals and objectives, as modest in some cases as they might be. 

Programs were attentive to the provision of information to students on: p lannin g for their 
careers, expectations for further vocational training; knowledge of workplace activities; 
and. integration of school and workplace learning and career interests. While some 
program directors ha\ e anempted significant linkages to vocational skill training programs 
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for their students once they had completed the ABE progr am^ not all are doing this Many 
need to strengthen efiforts in these areas and have requested assistance in developing 
strategies for networking with other service providers, school and agencies in their areas. 
It is critical to cormect students to employment as part of any adult education program. 
Some of these stepping-stone opportunities have resulted in skilled jobs for these students 
in such fields as medical transcription and other computer-related emplo\ment. 

One of the goals of any adult education program is clearly fixrther credentialing, fixture 
employment or fixrther education and training. Adult education programs are the natural 
bridge from illiteracy, welfare dependency, and unemployment to productive participation 
in society. 

Recomnt en clati ons: 




O Utilize student assessment data to measure improvements in student 
achievement. 

© Strengthen the integration of school and workplace learning and 

develop bridges and transitional programs from local adult education 
programs to programs wliich will lead to fixrther credentialing, 
future employment or fixrther education and training. 

© Coordinate with other local and county service providers including 
other social service agencies, vocational technical schools, the 
Work Investment Boards (WIBs) and other community-based 
orgarxizations. 



j 

i 

4 



Program Planning 

Indicator 3: Based on a participatory planning process, the program develops, 

implements, evaluates and revises, as needed, written program goals. 




21 
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Findings: 

A review of the planning documents supplemented by discussions with program directors 
and their staff revealed that most directors utilized their state approved grant award as 
their formal planning document. In very few instances were other comprehensive plans or 
strategic plans available which incorporated adult education programs into a more holistic 
view for the sponsoring agency. A few larger community-based organizations deriving 
funding from multiple government, state and/or private foundations and agencies utilized a 
more rigorous and comprehensive planning process which outlined an organizational 
mission statement, philosophy, goals and objectives and strategies to accomplish the stated 
goals. In those cases, the state-fimded adult education programs were subsumed in the 
overarching mission of the organization. In these cases the resources of the organization 
were also utilized to support the needs of adult students enrolled in either ABE or ESL 
programs. These ancillary services included counseling and other social services support, 
immigration information and job counseling and placement assistance. 

Wlien adult education programs are intemvined into the fabric of a larger organizational 
entit\. the o\ erall program services are more comprehensive and are more supportive of 
student needs. In some cases, adult directors reported that the adult education programs 
were not priority programs in the sponsoring agencies. On the other hand, some of these 
agencies provided strong financial support and organizational commitment to adult 
programs through, for example, purchasing of computers, setting up learning labs and/or 
seiv ing as a site for the One-Stop Career Centers. If adult programs are a part of an adult 
liigli school, vocational school, postsecondary institution, library, community learning 
center, or other service provider, etc. there generally is greater support and commitment 
to the programs and consequently to the students. The small adult education programs 
seiving 50 to 75 students which were independent of any other program connection were 
operational with commitment of that particular program director to serving the needs of 
this population. 

Most programs complied with the state requirement of maintenance of an advisory coimcil 
and sought input from the council on a regular basis. In a few cases, programs did not 
have a formal advisory cotmcil. Large community-based organizations utilized their 
boards of directors or trustees for advice and input to their programs. For small programs 
having a part-time staff of ten or less, a great reliance was placed on informal meetings 
and discussions to review program content, delivery and student issues. 

Program directors also varied m their approaches to gathering feedback from other 
sources additional to their advisory councils. Some program directors had rather 
comprehensive student course evaluations in place. In other cases, formal input from 
students as to their reactions to the program was non-existent. Though it should be said 
that individual teacliing staff members indicated that they always were in touch with the 
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needs of their students and consequently this informal data is certainly available for 
planning purposes. 



Recommendations: 



Develop a formal, integrated plan in order to deliver a more cohesive and 
comprehensive program that allows for a continuum of services for adult 
education students who have completed program requirements and are 
ready to move to employment or a more advanced level of education or 
vocational training. 

Create local task forces, advisory' councils, or sub-committees of 
larger groups, e.g., Workforce Investment Boards (WEBs), in order to 
develop a coimnon vision among coordinating agencies and service 
providers. 



Indicator 4: The program has curriculum and instruction sensitive to individual 
student and group learning styles and the varying levels of student need. 



Data from the individual student ponfohos which document student skill levels and 
educational needs form the foundation upon which to base delivery of curriculum and 
instruction. The focus of instruction centers around student needs and is frequently fine- 
tuned to respond to immediate and emerging student issues. In that sense the curriculum 
responds to life skill and enqiloyability issues. 

Programs have a formal curriculum guide in place or are using either or both of the two 
state-developed curricula. Adult Basic Skills Curriculum Guide (1988) and Curriculum 
Guide for English as a Second Language Programs and Evening Schools for Foreign 
Born Residents (1988), as their framework(s). Teaching staff exercise fiexibihty and 
mnovation in supplementing either locally developed guides or the state guides with 
materials of their own development or choosing. Teachers have found useful and practical 
tools amidst the myriad of materials produced by national text pubhshers. Where resources 
permit, teaching staff are using the computer as a tool to reinforce basic skills or in the 
teaching of English. 




Curriculum and Instruction 



Findings: 
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Teaching staff face many additional pedagogical obstacles in working with adult students. 
Adults may manifest some additional le arnin g or developmental disability which impacts 
their ability to learn. Providing appropriate educational strategies for these adults is 
complex in the context of a class. Staff in the ESL program face the additional challenge 
of immigrant students coming from many different language backgrounds, in addition to 
non-literacy of these students in their native languages. Staff must then provide 
instruction in a multi-level classroom setting. 

Program staff also indicated that volunteers supported their program and these volunteers 
provided additional tutoring support and time to students to supplement their classroom 
learning. 

Recommendations: 

O E.xpand utilization of curriculum and approaches to instruction in the multi- 
level classroom. 

© Review recent adult education curricula and materials that will enhance 
instruction and promote student learning. [Materials are available at the 
Adult Literacy Enhancement Center (.ALEC).] 



Staff Development 

Indicator 5: The program has a staff development process that considers the 

specific needs of its staff, offers opportunities for training in the skills necessary to 
provide quality instruction and for the utilization of skills learned. 



Findings: 

Based on the interview data, staff development was an area which directors and their staff 
viewed as vital in terms of continual professional development and renewal Staff 
development activities, including presefvice and inservice opportunities, were included in 
the overall program plan of operation but not formally link ed to program improvement 
and evaluation activities. In most cases, either a formal or informal p lannin g mechanism 
existed for the articulation of staff professional development needs. 
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Although formal and informal pl annin g for professional development had taken place, it 
became clear that staff development was an area of great need. Some directors and 
teachers indicated that they participated in staff meetings and in-service tra inin g provided 
in-house by their school district or respective agency. Participation of directors and staff 
in externally provided seminars, workshops and training sessions was marginal. There 
were many reasons cited for lack of participation. Principal among these is the fact that 
many adult education staff are part-time staff who hold full-time jobs in the daytime and 
yet still make the commitment to teach two to three times per week. Their ability to 
commit another evening or Saturday to attend a training session in their words was 
additionally hampered by the driving time and location of the training opportunities. They 
felt much more inclined toward attending training sessions either on-site or in regional 
locations. 



From an administrative standpoint, directors were also reluctant to e.xiend professional 
time to their staff to attend a seminar since this would necessitate either canceling class or 
tiuaing a substitute for that class. Directors advised that it was very difiScult to find 
substitute teachers for adult programs, particularly for the ESL program. Some directors, 
liON'. ever. from larger adult education programs have been \ ery supportive of their staff 
ana lia\ e facilitated their attendance at numerous tra inin g activities ineludina those 



si)o:'.sored by the Adult Literacy Enhancement Center. Directors indicated that they 
we;comed the inclusion of federal dollars targeted to staff ie\ elopment in the 1997-1998 
Request for Proposals for the Adult Basic Skills Grant Program. 



Directors who participated in the study had a formal staff performance appraisal system in 
place and indicated that they observed staff at least once per year if not more. In most 
cases, administrators and staff operate on a collegial and informal basis. Staff indicated 
that the program administrator made numerous visits to respective classes either to offer 
guidance or to counsel students throughout the year. These visits were viewed to be 
sup port we and not obtrusive or negative. 



Recommendations: 



O Involve staff in the identification and prioritization of professional 
development tra inin g needs. 

© Explore alternatives for the provision of staff development and 

training such as instructional television (ITV), computer linkages, 
internet access, web sites and chat rooms. 




O 



Link staff development activities with program evaluation and 
improvement. 



O 
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O Establish a system to provide ongoing professional development 
opportunities for staff 

® Provide local adult program staff with opportunities to interact and 

to exchange knowledge with colleagues in the field. 

® Utilize staff development fimds to enhance areas such as: working with 

adults with disabilities, femily literacy, life and employability skills, citizen- 
ship preparation, integrating technology into the classroom, and 
computerized recordkeeping. 



Support Services 

Indicator 6: The program identifies support service needs and the agencies which 
address these needs, provides information to students as needed, and promotes 
student access to these agencies through informal linkages with service providers as 
a means of enhancing student participation in the program; 




Findings: 

As has been discussed pre\iously, adults enrolled in ABE and ESL programs face many 
barriers to participation including the need for child care, counseling and transportation. 
Wliile some ma>' be employed, they in fact might have two marginal jobs and consequently 
have associated problems of finding appropriate child care and adequate personal time for 
class panicipation. The lack of transponation is another hardship faced by smdents. 
Program staff haye indicated that in some cases they even pick up and drop off their 
students to ease the lack of transportation burden. 

Program participants are often additionally burdened by substance, spousal or sexual 
abuse; family problems; social issues; or health concerns. As discussed earlier, a large 
number of program participants are not enq>loyed and consequently they confront welfare 
issues and in some cases homelessness. Other adults are immigrants or refugees who must 
deal with the newly enacted immigration reforms and the ensuing concerns brought about 
by these reforms. 

Many adults enrolled in programs may also have learning disabilities. Nightingale et aL 
(1991) estunate that between 20 and 30 percent of economically disadvantaged adults— 
and as many as 50 to 80 percent of ABE participants— may be learning disabled. 
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Given the multi-dimensional problems &ced by students in adult progr ams , the provision 
of support services is critical in order to sustain class participation. The identification of 
appropriate support services providers can be more easily acconq)Iished through 
coordination and networking. Directors may have worked in the adult program in their 
community for many years and consequently they have made extensive for mal and 
informal contacts throughout the community with other agencies and social service 
providers. In most cases, staff are also equipped to provide assistance and guidance to 
students in terms of referrals to agencies in the community which would help to address 
their needs. 

Directors who could offer additional support services, such as counseling, child care, 
health services, or legal information, to their students either on-site or through referrals to 
other resources, felt that they were in a better position to retain students in their programs. 



Recommendations: 

O Identif>- appropriate support service providers for program participants 
tlirough coordination and networking . (The need for child care, 
coxmseling and transportation is especially critical to progi-am 
participation.) 

© Provide increased counseling services to students in the program. 

© Identify other service providers in the area with whom formal and informal 

agreements can be structured to leverage resources and support for 
students in need. 

O Identify job opportunities as well as additional training and/or educational 
opportunities for post program participation. 



Recruitment 

Indicator 7: The program recruits the population in their service delivery area as 
identified in the Adult Education Act as needing literacy services. 
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Findings: 

Recruitment to the program does not seem to be an issue based on the findings of the 
study. Programs may operate on an open entry/open exist basis. Generally, fo rmal 
brochures are mailed to the resident community' two to three times per year. These formal 
mailings are supplemented by announcements posted throughout the community and 
neighboring co mmuni ties, public cable spots, flyers sent home with students and posted in 
workplaces. Sometimes these announcements and flyers are made ancL'or printed in 
languages other than English to reach the targeted clientele. Often program information is 
provided informally through word of mouth by current or previous program participants. 

In some commumties the program serves students fi'om neighboring towns ghen the 
location of the program and the hours the program is offered. Directors have indicated 
that programs ma\' have waiting lists and cannot acco mm odate all of the potential students 
who are interested in participating in the program. This is especially true of ESL 
programs. 

Wliile programs at present may not target special recruitment efforts to welfare recipients, 
welfare reform is increasingly linking recipients with adult education programs and 
programs providing GED attainment through increased networking and cooperation. It is 
estimated that about two-thirds of Aid to Families with Dependent Children (.APDC) 
recipients in Job Opportunities and Basic Skills (JOBS) programs require basic skills 
training before mo\ing into the work force (Martinson and Friedlander. 1994). 

Recommendations: 

O Prepare recruitment strategies targeted to the different populations of 
students served by the local adult education program. 

P Prepare recruitment materials wWch can be used with area employers to 
generate interest in the local workplace literacy program. 

© Prepare recruitment strategies and materials that target welfare recipients. 



Retention 

Indicator 8: Students are encouraged to remain in the program long enough to 
meet their educational goal(s). The program has a plan which addresses retention 
and implements the plan as needed. 

I 
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Findings: 

Retention is a problem faced by adult education program directors. Data from this study 
support national trends which suggest that adults do not stay in adult education progr ams 
very long. As Moore and Stavrianos (1995) point out, the median exposure is about 60 
hours of instruction over a period of 24 weeks with ESL students having a generally 
higher rate of program participation. ESL chents on the average accumulate substantiallv 
more hours of instruction. 

Tliere are man\ factors which influence participation and retention in adult education 
programs. Tliese factors include program characteristics, participant characteristics and 
participant moth ation. 

Moore and Sta\rianos also (1995) report that lower drop-Out rates tend to occur in 
programs that; 



■ offer a broad array of support services and emphasize integration of 
services with other agencies. 

3 de\ ote relatively few resources to client outreach and recruitment. 

S have a moderate or low cost per chent seat hour. 

■ are not extremely large as measured by the number of clients they serve. 

That little outreach should reduce drop-out rates is at first surprising, since these factors 
suggest a non-intensive intenention. Programs that emphasize client outreach and 
recruitment, however, may email less interested clients, who would in turn be more likely 
to drop out (Ivloore and Stavrianos, 1995 ). 

In addition it should be pointed out that progr^ retention is enhanced by the availabflity 
of support services noted previously, such as child-care, transportation and coimseling. 
Programs may also use college students or high school honor students as volimteer tutors 
who work with adult students for an hour before class either on a one-on-one basis or in 
small groups. Directors indicated that having a wider array of options for class hours of 
instmction would enhance student participation. Day-time programs rather than 
evening programs also seem to be related to more persistent participation. 
Participation in programs is also related to socioeconomic, demographic and educational 
variables as well as to opportunities for further education and tr ainin g leading to 
employment. 
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Motivation of adult students is another variable to participation. Motivational fectors 
enhancing participation include educational advancement, self-development and economic 
self-sufl5ciency and improvement. 

Recommendations: 



Offer a broad array of support services to students, either on-site 
or through coordination with other resources. 



© Emphasize integration of services with other agencies. 

© Offer a wide variety of options in terms of class hours of instruction (i.e. 
day, evening and Saturday classes). 



Connect completion of adult training to fixture employment opportunities 
or to fiirther education and training. 




e 



Establish structured policies and requirements regarding student attendance 
and commitment. 



Conclusions: 



Tlie field data strongly suggest that the eight indicators of program quality and the 
ancillary measures and performance standards form the firamework of most, if not all, adult 
education programs in the state. The self-study dimension of this field-based program 
review allowed for program directors and their stafis to review individual programs using 
the eight quahty indicators as a template against which to measure their individual 
program components, delivery of services, success and impact on the clients and overall 
achievement of their mission and program goals. Even though inqxlementation of these 
indicators varies, it is clear that program directors have been striving to make their 
programs more responsive to student needs and to adhere to sound management practices 
in attenqxts to improve the overall quahty of programs for students. The conclusions of the 
study which are delineated below support and are reflective of national trends (see 
Appendix C). 



O Programs that are part of a coordinated service delivery network and those 
which offer a continuum of comprehensive services are more effective and efficient than 
smaller programs which are unable to offer the wide range and scope of services necessary 
to meet the \ aried and often complex needs of adult students. 
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® Adult education program directors need to strengthen efforts to coordinate 
with agencies such as Workforce Investment Boards (WIBS), One-Stop Career Centers, 
and local or county welfare and workforce development service providers to ensure more 
comprehensive results-oriented services. 

© Comprehensive planning in a formal, integrated manner, consisting of an 
organizational mission statement, philosophy, goals and objectives and strategies to 
accomplish the stated goals, is necessary in order to ensure the most successful delivery' of 
coordinated services. 



O Impro\ements are necessary in data collection methods, data analysis and 
in the quality of program data that are obtained in order to demonstrate program results, 
panicularly progress and gains in student acliievement. 



® histmction must include curricula which place a greater emphasis upon 
career infoiTnation. employability skills and workforce development to prepare students 
for employment and occupational success, more advanced levels of education or 



\ ocational trainhis. 



© A wide range of options in available hours for classroom instruction, 
especially daytime hours, should be offered to assist students in meeting their educational 
goals. Daytime instruction seems to be related to increased and persistent paiiicipation and 
retention in adult education programs. 



© Directors should e.Kplore new technologies for program delivery, including 
distance learning, state-of-the-art software, and computer labs for strengthening students' 
computer literacy skills, immediate access to current information, and for instructional 
purposes. 



0 Local program staff development efforts and funds should be targeted to 
traming designed to unprove student retention, achievement and perfor man ce, and should 
focus upon areas such as family literacy, adults with disabihties, life and employabihty 
skills, citizenship preparation, integrating technology into the classroom and computerized 
recordkeeping. 



These conclusions support the initial phase of a restructured delivery system for adult 
education that has been introduced through the 1997-98 Adult Basic Skills Request for 
Proposal (RFP). The new guidelines strive to eliminate the fragmentation that has been 
evident for many years in adult programs. Research has indicated that on a national level, 
as well as in New Jersey, adult education programs have traditionally been provided in a 
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fragmented manner, and that this fragmentation has adversely mq>acted results. The focus 
of this RFP system is on the provision of a continuum of comprehensive adult education 
programs and services that: integrates essential components; coordinates activities with 
other state, federal and local efforts, such as School-to-Work, Workforce Investment 
Boards (WIBs), welfare reform efforts, the One-Stop Career System network; fosters the 
use of technology; and encoiuages parmerships with business and industry. 
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Appendix A: Program Sites Visited 
1996-1997 



County 


LEA 


Agency 


Program 


1 


120 


Atlantic County Vocational 


ABE & ESL 


1 


4180 


Pleasantville 


ABE & ESL 


2 


4019 


Atlantic Community College 


ESL 


3 


285 


Bergen County Special Services 


ABE 


3 


290 


Bergen County Vocational 


ABE & ESL 


3 


1370 


Englewood 


ABE & ESL 


3 


1450 


Fair Lawn 


ABE & ESL 


j 


1760 


Glen Rock 


ESL 


4 


4021 


Bergen Community College 


ABE & ESL 


4 


6011 


Associated Craftsmen/Easter Seals 


ABE 


5 


4050 


Pemberton 


.ABE 


5 


4320 


Rancocas Valley Regional 


ABE & ESL 


7 


390 


Black Horse FHke Regional 


ABE 


7 


680 


Camden 


ABE & ESL 


7 


4060 


Peunsauken Township 


ABE 


8 


400' 


Camden County College 


ABE 


9 


720 


Cape May Coimty Vocational 


ABE & ESL 


9 


5790 


Wildwood 


ABE & ESL 


1 1 


540 


Bridgeton 


ABE & ESL 


1 1 


5390 


Vineland 


.ABE & ESL 


12 


3054 


Cumberland Rehabilitation Center 


ABE 


13 


660 


Caldwell-West Caldwell 


ABE & ESL 


13 


1210 


East Orange 


ABE & ESL 


13 


1387 


Essex Coimty Education Services 


ESL 


13 


1390 


Essex County Vocational 


ABE & ESL 


13 


2330 


Irvington Township 


ABE & ESL 


13 


3570 


Newark 


ABE & ESL 


13 


3880 


Orange 


ESL 


13 


4900 


South Orange-Maplewood 


ABE & ESL 


14 


4022 


Essex County College 


ABE 


14 


6016 


First Occupational Center 


ABE 


14 


6022 


Catholic Community Services 


ESL 


14 


6137 


Focus Newark, Inc. 


ABE & ESL 


14 


6231 


Jewish Vocational Services 


ABE & ESL 


16 


4022 


Gloucester County College 


ABE & ESL 


17 


2210 


Hoboken 


ABE & ESL 


17 


2390 


Jersey City 


ABE & ESL 


17 


4730 


Secaucus 


ABE & ESL 



O 



28 



33 



County 


LEA 


Agency 


Program 


17 


5240 


Union City 


ABE & ESL 


18 


2003 


International Institute 


ESL 


18 


6257 


ILGWUNJ Region 


ESL 


19 


2305 


Hunterdon County Educational Services 


ABE & ESL 


21 


1245 


East Windsor Regional 


ABE & ESL 


21 


1950 


Hamilton Township 


ABE & ESL 


21 


5715 


West Windsor-Plainsboro Regional 


ESL 


22 


4023 


College of New Jersey 


ABE & ESL 


22 


4024 


Mercer County Commimity College 


ABE 


22 


6014 


Princeton YWCA 


ESL 


23 


1170 


East Brunswick 


ABE & ESL 


23 


1290 


Edison 


ABE & ESL 


23 


3530 


New Brunswick 


ABE & ESL 


23 


4090 


Perth Amboy 


ABE & ESL 


23 


4860 


South Brunswick 


ABE & ESL 


23 


4910 


South Plainfield 


ABE & ESL 


23 


4920 


South River 


ABE & ESL 


23 


5850 


Woodbridge 


ABE & ESL ' 


24 


4031 


Rutgers Unh'ersity 


ESL 


24 


6017 


Raritan Valley Workshop/Easter Seals 


ABE 


25 


100 


Asbury Park 


ABE & ESL 


25 


3255 


Monmouth- Ocean Educational Services 


ABE & ESL 


25 


3260 


Monmouth County Vocational 


ABE & ESL 


26 


3067 


Sepharciic Russian Resettlement 


ESL 


26 


3068 


Cerebral Palsy-Monmouth & Ocean 


ABE 


26 


4008 


Brookdale Community College 


ABE & ESL 


27 


3365 


Morris County Vocational 


ABE & ESL 


27 


3370 


Morris Hills Regional 


ESL 


27 


3385 


Morristown 


ESL 


27 


3950 


Parsippany-Troy Hills 


ABE & ESL 


29 


530 


Brick 


ABE & ESL 


29 


770 


Central Regional 


ABE & ESL 


29 


2520 


Lakewood 


ABE & ESL 


29 


5190 


Toms River Regional 


ABE & ESL 


31 


3970 


Passaic 


ABE & ESL 


31 


3995 


Passaic County Vocational 


ABE & ESL 


31 


4010 


Paterson 


ABE & ESL 


34 


3036 


ARC of Salem 


ABE 


35 


1610 


Franklin 


ABE 


36 


6028 


Jointure for Community Adult Ed. 


ABE & ESL 


38 


4010 


Sussex County Commimity College 


ABE & ESL 


38 


6011 


Highlands Workshop/Easter Seals 


ABE 


39 


1320 


Elizabeth 


ABE & ESL 



29 



34 



County 


LEA 


Agency 


Program 


39 


4160 


Plainfield 


ABE &. ESL 


39 


4290 


Rahway 


ABE &, ESL 


39 


4540 


Roselle 


ABE 


39 


4550 


Roselle Park 


ESL 


41 


5460 


Warren Couniy Vocational 


ABE &. ESL 


42 


4040 


Warren Couniy Community 


ESL 





Number of Programs Offering both ABE and ESL 


56 


Number of Programs Offering ABE only 


16 


Number of Programs Offering ESL only 


15 


Total Number of Programs Visited 


87 



Appendix B: Interview Resppnses 
Self-Study Guide 

Indicators of Program Quality, Measures of Performance and 

Standards 

1996-1997 



Indicator 1: Learners demonstrate progress toward at tainment of basic skill 
competencies that support their educational needs and goals. 



MEASURE OF PERFORMANCE 



1. 1 Do you have standardized test score gains for each Level II and Level III student 
in a portfolio? 



Yes 


70 


No 


3 


Unable to Ascertain 


14 



1.1.1 Are all Students assessed with a state-approved standardized reading 
comprehension test witliin ten hours of enrollment? 



Yes 


70 


No 


1 


Unable to Ascertain 


16 



1. 1.2 Are all students reassessed on a standardized reading comprehension test after 
being in attendance for 50 hours of instruction? 



Yes 


67 


No 


2 


Unable to Ascertain 


18 



1. 1.3 Do 60 percent of the students in attendance for 50 hoiurs of instruction show the 
equivalent of a six-month gain expressed as a scaled-score equivalent on a standardized 
reading comprehension test? 



Yes 


45 


No 


9 


Unable to Ascertain 


33 




O 

ERIC 



36 



MEASURE OF PERFORMANCE 



1.2 Do you have informal reading inventory results, pre-post, for each Level I student 
in a portfolio? 



Yes 


66 


No 


5 


Unable to Ascertain 


16 



1 .2. 1 Are all students assessed with an informal reading inventory within ten hours of 
emollment? 

Yes 69 

No 4 

Unable to Ascertain 14 

1.2.2 Does documentation exist relative to the achievement of an individual stated 

objective or growth on an informal reading inventory after a student has been in 
attendance for 50 hours of instruction? 




Yes 


65 


No 


3 


Unable to Ascertain 


19 



1.2.3 For those students in attendance for 50 hours of instruction, do 60 percent show 
either gain on an informal reading inventory or attainment of some individual stated 
objective? 



Yes 


57 


No 


6 


Unable to Ascertain 


24 



MEASURE OF PERFORMANCE 



1.3 Do you have placement test scores for each ESL student in a portfolio? 



Yes 

No 

Unable to Ascertain 



71 




32 



37 



16 



1.3. 1 Are all ESL students assessed with a state-approved placement test wi thin ten hours 
of enrollment (contact)? 



Yes 


68 


No 


1 


Unable to Ascertain 


18 



1.3.2 Does documentation exist relative to the achievement of a personal instructional 
objective for each student in attendance for 50 hours of instruction? 



Yes 


65 


No 


2 


Unable to Ascertain 


20 



1.3.3 For those students in attendance for 50 hours of instruction, do 75 percent repon 
the achievement of a personal instructional objective? 



Yes 


55 


No 


5 


Unable to Ascertain 


27 



MEASURE OF PERFORMANCE 



1.4 Do you have teacher reported gains and improvements in computation and/or 
communication and/or life skill competencies (employability skills) in a portfolio for . 
each student? 



Yes 


71 


No 


4 


Unable to Ascertain 


12 



1.4. 1 Does documentation exist relative to gains and improvements in confutation and/or 
communication and/or life skill competencies as evidenced by teacher notation on 
educational plans for all students in attendance for 50 hours of instruction? 



Yes 

No 

Unable to Ascertain 



70 

3 

14 



1.4.2 For those students in attendance for 50 hours of instruction, do 75 percent show 
gains and improvement in conq)utation and/or communication and/or life glfill 
competencies as evidenced by teacher documentation on educational plans? 

Yes 60 

No 9 

Unable to Ascertain 18 



1.4.3 Do you have documentation available for each student in a portfolio which relates 
growth in computation and/or communication and/or life skill conqjetencies (employability 
skills) to a student's career goals/school-to-work transition? 

Yes 68 

No 4 

Unable to Ascertain 15 



MEASURE OF PERFORMANCE 



1.5 Do you have individual student goals, skill needs, student comments and feedback 
and examples of student work in a portfolio for each student? 

Yes 81 

No 3 

Unable to Ascertain 3 

1.5.1 Does the educational plan reflect student goals and identify student skill needs 
based upon assessment? 

Yes 81 

No 3 

Unable to Ascertain 3 



1.5.2 Are examples of student work maintained in a student folder? 



39 

34 



Yes 

No 

Unable to Ascertain 



77 

3 

7 



1.5.3 Are student comments and feedback related to program participation and gains 
included in the student folder when available? 



Yes 69 

No 

Unable to Ascertain 






40 



35 



VO VO 



Indicator 2: Learners progress according to their abilities in the instructional program 
or complete program educational requirements that allow them to continue their education 
or training or achieve their personal educational goals. 



MEASURE OF PERFORMANCE 



2. 1 Do you have evidence of students' advancement to a higher level of skill or 
competency in the adult education program? 

Yes 78 

No 1 

Unable to Ascertain 8 




2. 1. 1 Does documentation exist relative to the improvement of skills for all Level I 
studeuts. including special needs students, in attendance for 50 hours of instruction in 
Level I programs as evidenced by educational plans? 

Yes 75 

No 3 

Unable to Ascertain 9 



2. 1.2 For Level I students, including special needs students, in attendance for 50 hours of 
instruction in Level I programs, do 40 percent show improvement of skills as evidenced by 
their educational plan? 



Yes 


62 


No 


5 


Unable to Ascertain 


20 



2. 1.3 For Level I students, with the exception of special needs students, do an additional 
ten percent move to Level II? 



Yes 


54 


No 


6 


Unable to Ascertain 


27 



2. 1.4 Does documentation exist relative to the improvement of skills for all Level II 
students in attendance for 50 hours of instruction in Level II progr ams as measured bv 
performance on an approved test? 



Yes 


72 


No 


4 


Unable to Ascertain 


ll 




O 
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2.1.5 For Level II students in attendance for 50 hours of instruction, do 40 percent show 
improvement of skills within Level II as measured by performance on an approved test? 



Yes 

No 

Unable to Ascertain 



59 

5 

23 



2. 1.6 Do an additional ten percent of Level El students move to Level HI? 



Yes 


55 


No 


5 


Unable to Ascertain 


27 



2.1.7 Does 


your program provide information to students on: 


□ 


planning for their careers; 


□ 


e.xpectations for further vocational training; 


□ 


knowledge of workplace activities; and, 


□ 


integration of school and workplace learning 
and career interests? 




Yes 79 

No 

Unable to Ascertain 8 




\measure of performance 



2.2 Do you have evidence that students are attaining stated personal educational 
goals? 



Yes 

No 

Unable to Ascertain 



76 

3 

8 



2.2. 1 Does documentation exist relative to the achievement of a stated personal objective 
as part of a long-range educational goal for each student in attendance for 50 hours of 
instmction? 



Yes 


68 


No 


2 


Unable to Ascertain 


17 





37 



42 



# 



2.2.2 For those students in attendance for 50 hours of instruction, do 75 percent achieve a 
stated personal objective as part of their long-range educational goal? 



Yes 


61 


No 


S 


Unable to Ascertain 


21 




43 

38 




Indicator 3: Based on a participatory planning process, the program develops, 
implements, evaluates and revises, as needed, written program goals. 



MEASURES OF PERFORMANCE 



3. 1 Do you have a planning document that specifies program goals and objectives and 
is regularly reviewed and revised at least once each year? 

Yes 81 

No 3 

Unable to Ascertain 3 



3.2 Is your program open to community input on planning through mechanisms such 
as an advisory board, staff meetings or comments, student questionnaires or public 
hearings? 

Yes 77 

No 4 

Unable to Ascertain 6 



3.3 funding constraints permit, does your program plan respond to and address 
community needs and priorities regarding location of classes, skills taught, and type of 
program services offered? 



Yes 79 

No 

Unable to Ascertain 8 



3.4 Do you have an evaluation system in place that feeds into the planning process? 



Yes 

No 

Unable to Ascertain 



84 

2 

I 



3.5 Are activities which are implemented based on reviewed and revised program goals 
and objectives? 



Yes 

No 

Unable to Ascertain 



82 

2 

3 




39 



44 



3.5.1 Do you have a plan which is reviewed and approved annually and consists of the 
following: 

□ Community needs assessment t^ch includes, but is not limited to 
current demographics (community resources, trends, and projections); 

□ Student skill assessment (results of formal/informal assessment); 

□ Input and participation of staf^ advisory group, and program parti- 
cipants; 

□ Documentation that appropriately trained and/or certificated staff are 
utilized for instruction; 

□ Annual review of facilities documenting compliance with local 
health and safety requirements; 

□ Review of an interface with district's school-to-work plan; 

□ Goals and objectives; 

□ Implementation plan; and 

□ Evaluation? 



Yes 

No 

Unable to Ascertain 



74 

4 

9 



Indicator 4: The program utilizes curriculum and instruction techniques sensitive to 
individual student and group learning styles and the varying levels of student need. 

MEASURE OF PERFORMANCE 



4.1 Is student assessment information used to guide the goals and design of the of the 
instructional process? 

Yes ' 86 

No 

Unable to Ascertain / 



Is a student portfolio which reflects student need used as a basis for instruction? 



Yes 85 

No . 

Unable to Ascertain 2 



MEASURE OF PERFORMANCE 



4.2 Do you have a written curriculum guide outlining instructional content including 
materials, approaches and strategies to address the educational needs and goals of 
individual students? 



Yes 81 

No 2 

Unable to Ascertain 4 



4.2. 1 Do you use the cumculum guide as a basis to address the educational needs and 
goals of individual students? 



Yes 

No 

Unable to Ascertain 



84 

2 

I 



# 



Indicator 5: The program has a staff development process that considers the specific 
needs of its stafE^ offers opportiuuties for training in the skills necessary to provide quality 
instruction and for the utilizatibn of skills learned. 



MEASURE OF PERFORMANCE 



5. 1 Do you have preservice and inservice staff development opportunities that include 
a program overview, philosophy and goals of the program, and ongoing topics 
appropriate to adult learning? 

Yes 83 

No 1 

Unable to Ascertain 3 



5.2 Do you have a process for identifying staff development needs? 



Yes 

No 

Unable to Ascertain 



79 

3 

5 



5.3 Do you have effective staff performance assessment in place as measured by 
administrative observation of staff? 



Yes 

No 

Unable to Ascertain 



83 

I 

3 



5.3.1 Do you have a staff development plan which is reviewed and approved annually and 
consists of the following: 

□ Needs assessment; 

□ Goals and objectives; 

□ Implementation plan; 

□ Budget and resources: and 

□ Evaluation? 



Yes 

No 

Unable to Ascertain 



65 

16 

6 




n 

t 



42 



MEASURE OF PERFORMANCE 



5.4 Do you have a record of the hours of preservice and inservice staff development 
training received by staff? 



Yes 


63 


No 


14 


Unable to Ascertain 


10 



5.4. 1 Do you have documented evidence of staff development activities and tr ainin g (i.e. 
minutes, workshop agendas, attendance, records, etc.)? 



Yes 


69 


No 


8 


Unable to Ascertain 


10 



48 



43 



Indicator 6; The program identifies support service needs and the agencies AAhich 
address these needs, provides mformation to students as needed, and promotes student 
access to these agencies through informal linkages vvith service providers as a means of 
enhancing student participation in the program. 



MEASURE OF PERFORMANCE 



6. 1 Do you have a process for identifying student support service needs? 

Yes 77 

No 5 

Unable to Ascertain 5 



6. 1. 1 Do you have documentation available which reflects and supports the need for 
student-driven requests? 



Yes 

No 

Unable to Ascertain 



57 

21 

9 



MEASURE OF PERFORMANCE 



6.2 Do you have a listing of support service providers, or informal agreements or 
linkages between the program and support service providers? 

Yes 79 

No 3 

Unable to Ascertain 5 



6.2. 1 Do you maintain a current and appropriate list of co mmunit y services? 

Yes 80 

No 3 

Unable to Ascertain 4 



m 
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Indicator 7: The program recruits the population m their service delivery area as 
identified in the Adult Education Act as needing : literacy services. 



MEASURE OF PERFORMANCE 



7.1 Do you have different types of recruitment activities available if the program is 
under subscribed? 



Yes 


68 


No 


3 


Unable to Ascertain 


16 



7. 1. 1 Do you maintain and document outreach and recruiting activities appropriate to the 
target population? 



Yes 

No 

Unable to Ascertain 



71 

5 

11 




Indicator 8: Students are encouraged to remain in the program long enough to meet 
their educational goal(s). The program has a plan which addresses retention and 
implements the plan as needed. 



MEASURE OF PERFORMANCE 



8.1 Do you have a written statement on the expected level of student participation and 
the measures or activities to be implemented if attrition is above stated acceptable 
levels? 



Yes 


64 


No 


10 


Unable to Ascertain 


13 



8. 1. 1 Do you have evidence of the program's written policy concerning the e\"pected level 
ot participation and the procedures which will be implemented to address retention 
andy'or termination if anticipated levels are not satisfied? 



Yes 


63 


No 


10 


Unable to Ascertain 


14 




\measure of performance 



3 



8.2 Do you have evidence of the program implementation hours by the type of 
program? 



Yes 85 

No I 

Unable to Ascertain I 

8.2. 1 Do you maintain records for students enrolled in ABE and ESL programs detailins 
their hours of participation in the program? 

Yes 87 

No - 

Unable to Ascertain 



ME.ASURE OF PERFORMANCE 



51 
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8.3 Do you have the number of students who continue or return to the program in the 
next program year? 



Yes 76 

No 4 

Unable to Ascertain 7 



8.3.1 Does your recruitment and intake process identify continuing or returning students? 



Yes 

No 

Unable to Ascertain 



74 

6 

7 



Appendix C 

Building State Performance Measurement, Reporting and Improvement Systems 

National Institute for Literacy 
June, 1995 



A flexible framework for systemic reform is premised on an accoimtability system as a 
mecha^m for focusing on the results of investments in literacy and improving returns on 
those investments. It is a powerful tool for systemic change— a guide for states just 
beginning the change process and a reinforcement for those states already on the way. It 
builds capacity in four specific areas within state literacy systems. Components of the 
system recommended by NALS are summarized below: 



■ Development of an accountability system focused on state and national 
goals aimed at providing a comprehensive literacy system aligned to larger state and 
federal human resource and economic development goals. Tliis tyi)e of accountability 
system will allow for the measurement of progress toward goals requiring interagency 
cooperation, program collaboration, coordinated service delivery', and the integrated effort 
of all stakeholders in the svstem. 



■ Documentation of progress by measuring the outcomes of programs, not 
the program dej\ ery process. Enrollment data and other inputs, while important, do not 
delineate unpawt of the program or describe the results of the program in terms of the 
differences in srudent s lives, such as the attainment of the GED, panicipation in the 
workforce, being able to attain a level of functional literacy to allow for paiticipation in 
society or ever, day family living. 



■ Development of a management information system which enables the state 
and other agencies to streamline reponing and share information. Centralized infonnation 
about programs will make it possible for better planning and deUvery of services especially 
in light of weltare reform and efforts to assist those on public assistance to participate in 
the world of work and in the national economy. 



■ Provision of a system for continuous in^rovement of adult education 
programs. The system would allow program directors to determine how well their 
program is achieving anticipated results and would provide effectiveness information to 
state policymkers. Program strengths and weaknesses could be identified and the state 
could then disseminate proven practice and discontinue ineJBfective practice. 



53 
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